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Abstract 

A compound determinant identity for minors of rectangular matrices is 
established. As an application, we derive Vandermonde type determinant 
formulae for classical group characters. 

1 Introduction 

Cauchy (1812) [3] and Sylvester (1851) [12] established the following identity for an 
arbitrary square matrix (see pm pp. 99-131 of vol. I, pp. 193-197 of vol. II] or [131 
pp. 87-89]): 

Proposition 1.1. Let s and n be positive integers such that s > n. Let A = 
( a ij)i<i,j<s be an arbitrary square matrix of size s. We denote by Aj the submatrix 
{<H k ji)i<k,i<n of si ze n f° r two subsets I = {i\ < ■ ■ ■ < i n } and J — {j% < ■ ■ ■ < j n } 
of [s] = {1,2,..., s}. Then we have 



det (det A I J ) I J = (det A) U-lj , (l.l) 

where the rows and columns of the determinant in the left-hand side are indexed by 
n-element subsets /, J of [s] and arranged increasingly in the lexicographic ordering. 

Using this proposition, in [6] the second and third authors gave a simple proof to 
the fact that a determinant formed by multiple 2r'4>2r basic hypergeometric series is 
evaluated as a product of g-gamma functions [TJ H]. As limiting cases, the hyperge- 
ometric determinant formula includes the following determinant formulae for Schur 
functions GL n (A;a;) and symplectic Schur functions Sp 2n (A;x). (Similar formulae 
hold for classical group characters. See [5].) 



*Department of Mathematics, Faculty of Education, University of the Ryukyus, Nishihara, 
Okinawa 903-0213, Japan, ishikawa@edu.u-ryukyu.ac.jp 

' School of Science and Technology for Future Life, Tokyo Denki University, Tokyo 101-8457, 
Japan, mito@cck.dendai.ac.jp 

■^Graduate School of Mathematics, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8602, 
Japan, okada@math.nagoya-u.ac.jp 



1 



Proposition 1.2. Let s and n be positive integers such that s > n and let X = 
(xi, . . . ,x s ) be an s-tuple of indeterminates. For an n-element subset I = {i\ < 

). Then we have 



;i.2) 





(x k - Xj)(l - XjX k ) 



;i.3) 



where the row index A runs over all partitions whose Young diagrams are contained 
in the rectangular diagram ((s— n) n ), and the column index I runs over all n-element 
subset of [s]. 

The aim of this paper is to establish another formula (Theorem 11. 3p for com- 
pound determinant like the Cauchy-Sylvester identity. As a consequence, we can 
derive identities involving classical group characters (Theorem [32]) similar to Propo- 
sition 11.21 The identities for Schur functions and symplectic Schur functions gener- 
alize the formulae given in j2j Corollary 3.1 and Proposition 7.1], which are obtained 
as limiting cases of another hypergeometric determinant discussed in [2]. 

In order to report the main result, we introduce some notations and terminolo- 
gies. For any integers r and s, we use the symbols [r, s] = {r, r + 1, . . . , s} and 
[r] = [l,r]. If r > s, then we use the convention that [r, s] =0. If S is a finite set 
and r is a nonnegative integer, let ( 1 denote the set of all r-element subsets of 5*. 

For I = {ii < ■ ■ ■ < i r } and J — {ji < ■ ■ ■ < j r } in ('^) , we write I < J if there is 
an index k such that 

ik-i = jk-i, ik < jk- 

This gives a total order on ( ), which is called the lexicographic order. 

Let s and n be positive integers. Let Z s n denote the set of (weak) compositions 
of n with at most s parts, i.e., 

Z s ,n = {" = ("i, • • • , AO e % s I «i > 0, . . . , n a > 0, ui H h/i s =n}, 

and let Z° n denote the set of compositions of n with exactly s parts, i.e., 

Z s,n = il^ = (^i> • • • > Vs) € Z s | ui > 0, . . . , u s > 0, Ui H h/i s = n}. 

Then the cardinalities of these sets are given by 

"M'T 1 )- ^■*'--C + -:: , " 1 )-C=i> 

We introduce the following total order on Z s ^ n . For A, u e Z SjTl , we write A < if 
there is an index k such that 

Al = /il, • • • , Afc_i = Hk-li > l^k- 
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To each integer sequence \i = ...,/i s ) with < \ii < n for all i, we associate an 
n-element subset 6 s , n (/i) G defined by 



5 ,n(X> = LJ [(* - l ) n + !>(*- l ) n + A*» 



;i.4) 



i=l 



If t s>n is restricted to Z S;n , then it gives an injection 6 s>n : Z s n — > (} s ^), and one 

readily sees that, for A, /i G Z s ^ n , i S)n (\) < i s ,n{.^) if and only if A < yu. We may 
write i = L Stn for brevity if there is no ambiguity on s and n. 
For example, if s = 3 and n = 2, then 

Z 3 , 2 = {(2, 0, 0), (1, 1, 0), (1, 0, 1), (0, 2, 0), (0, 1, 1), (0, 0, 2)}, 

and the map 63^ defined in HI .41) is given by 



.3,2(2, 0, 0) = {1, 2}, .3,2(1, 1, 0) = {1, 3}, 63,2(1, 0, 1) = {1, 5}, 
63,2(0, 2, 0) = {3, 4}, 63,2(0, 1, 1) = {3, 5}, 63,2(0, 0, 2) = {5, 6}. 

This injection 63,2 can be visualized by the following picture: 
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Let A = (aij)i<i<M,i<j<N be any M x N matrix, and let / = {ii < - • • < i r } C 
[M] (resp. J = {ji < ■ ■ ■ < j r } C [N]) be a row (resp. column) index set. Let 
£ii,...,ir ^ en0 ^ e m atrix obtained from A by choosing rows indexed by / 



Jl,-Jr 



and columns indexed by J. If r = M and I = [M] (i.e., we choose all rows), then 



we may write Aj = A 



for A^ when there is no fear of confusion. Similarly 



we may use the notation A = A 1 



r for ALj, when r = N and J 



[N]. 



The purpose of this paper is to prove the following theorem and give an appli- 
cation to identities for classical group characters. 

Theorem 1.3. Let s and n be positive integers and A = (ay)i<i<s+ n -i,i<7<sn be 
an (s + n — 1) x sn matrix. We put 

^=( [s + ^ -1] ), e = {i s ,M\fiez s , n }, c° = K„WI^C-i}' 

(Hereafter we assume 1 = 6 SiTC unless s and n are explicitly specified.) Then we have 



det (det Aj) 



l€ll,j£C 



JJ det A 

Jec° 



;i.s) 



where the rows and columns of the matrix on the left-hand side are arranged in 
increasing order with respect to <. 
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For example, if n = 1, then 

K = C = {{!},... ,{,}}, C° = {[s]} 
and Equation (11. 5p trivially holds. If s = 1, then 

^ = C = {M}, C° = {[n]} 
and Equation (II. 5p is also trivial. If s = 2 and n = 2, then Equation (II. 5p reads 
/detA^ detA\ 2 3 detA\f\ 



det 



det Aj| det A]\ det A 



13 



13 
1 31 



det -^4^23 * det ^4.-^^^. 



ydetA^ det^jj det A 2 3 l J 

If s = 2 and n = 3, then Equation (I1.5P reads 

/det^|| det AgJ det AJi deti4jgj\ 

det detAH detAJII detail det^i 

det ^^23 det -^4^24 det -^4^45 det yl^^g 

VdetAil det^H detA?t| det A™/ 

If s = 3 and n = 2, then Equation (11.51) reads 



j + A 1234 . j t 4 1234 . j + A 1234 
aCT/1 1234 QeX/1 1245 QCT/i 1456- 



det 



/det 
det 

det A\i 
det Af 2 
det A\\ 



det A\l 
det A\l 
det A\\ 
det AH 
det Al A 3 

134 
l 13 



det A\l 
det AH 
det A\\ 
det AH 
det AH 



det AH det AH 

det AH det AH 

dety4|| det AH 

det Af 4 det Af 5 

det AH det A 2 3 l 



det AH\ 
det AH 
det AH 
det AH 
det AH 



\det Afl det AH det Af| det Aft detAf 5 det AH J 



det Aim ' det AHH ■ det A^jjg. 



We note that, as polynomials in indeterminates a^-'s, the degree of the left hand 
side of (II. 5p is equal to 

/s + n-T 

n ■ #Z Sjn = n 



n 



and the degree of the right hand side is equal to 



( a + n-l)-#Zj 



s,s+n— 1 



fs + n- 1} 



s + n - 2 
n — 1 



so that the degrees coincide with each other. 
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2 A Proof of the main theorem 



In this section we give a proof of the main theorem, i.e., Eq. (II. 51) . Our proof splits 
into the following three steps: 

1. Finding the irreducible factors of the determinant on the left-hand side of (jl.5p 
(Lemma 12731 . 

2. Proving Eq. (JT3J) up to constant (Lemma 12. 5p . 

3. Determining the constant (Lemma 12.71) . 



2.1 A partial order on the set Z SjU of compositions 

Let s and n be positive integers, and fix a positive integer k such that 1 < k < s. 
Hereafter we call k a color and write C = {1,2, ... ,s} (the set of colors). We 
introduce a partial order ^ on the set Z a>n of compositions as follows. For A and 
\x in Z Stn , we define A ^ p if \ < pi for all i ^ k. For example, if A = (2, 0, 1, 3), 
p = (2, 1, 2, 1) G Z^e and k — 4, then we have A ^4 /i since Aj < /ij for i 7^ 4. This 
gives a partially ordered set (Z S)n , ^k), and we write A -<k H if A p and A ^ /j. 
Throughout this section, we will follow the general notation concerning posets used 
in the book [II] . We see that (Z s n , is a graded poset with the rank function 

p( k \p) =n- p k =} j p i for p = (p h . . . , p s ) e Z StTl . 

i^k 

This poset (Z StU , -<k) has the minimum element p$ = (0, . . . , 0, n, 0, . . . , 0) in which 
n is at the /cth position. Let P^ = {/i G Z s n | p^(fi) = i] be the subset of elements 
of rank i. By definition, Z Sj „ = P ^ U • • • U P„ is a disjoint union, where P = 
{/io^} and Pn^ = {/i = (/ii, . . . , p s ) G Z s>n | pk = 0}. Note that Pn^ can be naturally 
identified with Z s _i jn . Thus we have #P ^ = 1 and #Pn = ( s+ ™~ 2 )- F° r example, 
the Hasse diagram of (Z 3>2 , ^1) is shown as Figure U and we have P (1) = {(2, 0, 0)}, 
P[ x) = {(1,1,0), (1, 0, 1)} and P 2 (1) = {(0, 2, 0), (0, 1, 1), (0, 0, 2)}. 
We put 

n-1 

pW=\JpM = Z s , n \pM, 

which is the set of elements p = (fix, . . . , p s ) in Z s n such that pk > 0. Then we have 
# p(fc) = _ (*+*-2) = (-+»-2). We define' a bijection r<*> : P« -)• Z° s+n _ 1 

as follows. To each /a G F(% we associate v = (yi, . . . ,u s ) — r^(p) defined by 

A*i if z = fc, ^ j\ 

/ij + 1 if i 7^ fc. 
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(0,2,0) 



(0,1,1) 



(0,0,2) 




(1,0,1) 



Ho =(2,0,0) 
Figure 1: Hasse diagram of (Z 3j2 , d^i) 



(2.2) 



The inverse map of associates to v £ s+n _ 1 a composition fi = . . . , fx s ) 
defined by 

{z/j if i = k, 

Vi — 1 if i 7^ k. 

We also introduce another map : — > ( _J) , which will play a crucial role in 
the following arguments. To each fi £ P^ k \ define (ft k \ix) £ Q^l) by 

# (^)= U {(*- i)™+^+i}- 

l<t<s 

In the above example where s = 3, n = 2 and = 1, we have 

P« = pW U P[ l) = {(2, 0, 0), (1, 1, 0), (1, 0, 1)}, 
^ 4 = {(2, 1,1), (1,2,1), (1,1, 2)}. 

The map defined in (12. II) is given by 

r«(2, 0, 0) = (2, 1, 1), r«(l, 1, 0) = (1, 2, 1), r«(l, 0, 1) = (1, 1, 2). 

And the map 

0(i) . p(i) (M) defined in Q is given by 
0«(2,O,O) = {3,5}, 0«(1,1,O) = {4,5}, 0«(1,O,1) = {3,6}. 
This map can be depicted as follows: 



(2,0,0) i y 
(1,1,0) ^ 

(i,o,i) ^ 
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Then one easily sees that the following proposition holds from the definition: 
Proposition 2.1. Fix a color k £ C . For each /i £ P", we have 
i(jm) n <^(/i) = 0, i(jm) U ^fr) = l (r^'f/i)) . 
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2.2 Vector notation 

We write \I\ for ^2 ieI i HI — {ii, . . . ,i p } G (^)- If / is a subset of S, we denote 

the complement of I in S by I (or more explicitly by S\I). 

For two subset I and J of [JV], we define e(I,J) G {1,0,-1} as follows. If 
/ fl J 7^ 0, then we define e(J, J) = 0. Otherwise, if J = {ii < • • • < i p } and J = 
{ji < ■ ■ ■ < j q }, then e(I, J) is the signature of a permutation which transforms the 
sequence . . . , i p ,ji, . . . ,j q ) in the increasing order. For example, if I = {3, 4, 8} 
and J = {1,6, 9}, then we have 

(T n A 3 4 6 8 9\ , . 4 

6(/,J) = sgn^ 3 4 8 x 6 9 J=(-1) =1, 

where 4 is the inversion number of the above permutation. 

Let s and n be positive integers, and let A be an (s + n — 1) x sn matrix. For 
Je(?) andXe (H),we put 

V J (A) = (detA / J ) /e([s+rl]) , (2.3) 

V*(A) = ((-^'^^^det^)^^^^ , (2.4) 

which are both ( s+ ™ _1 ) -dimensional column vectors, where the entries are arranged 
increasingly in the lexicographic order of indices. For example, if s — 3, n — 2, 
J = {1, 3} G and X = {4, 6} G , then 

Vj(A) = * (det det det A^ det A| det Ajj det A?f ) , 
V^(A) = *(det^ -det AH detA™ detA\j -det A 1 * detA\l). 

For two vectors v = (vi) Ie ^ a +n-i]y w = (wi) Ie ^[ a +n-i]y we write 

(v, w) = l VW = ViWi. 

H ls+ r 1] ) 

Using this inner-product notation, the Laplace expansion formula can be written as 

(Vj(A),V K (A)) = e(J,K) det A JuK , (2.5) 

for J G and K G If we take the above example J = {1,3} G (f) and 

X = {4, 6} G ('J), then we have 

(Vj(A),V K (A)) = det A\l det A lt - det A ll det ^46 + det A \t det ^46 

+ det Afl det ^46 - det A H det ^46 + det ^13 det A ll 
_ j f 4 1234 

The following proposition will be used to compute the determinant in this section. 
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Proposition 2.2. Let s and n be positive integers, and let ibean(s + n-l)xsn 
matrix. Fix a color k G C. Let A G Z sn and /i G P^l Then we have 



<V t(A) (A),V^, M (A)) = 0, 



unless A /i. 



Proof. Assume A 2^ /1. Then there exists I 7^ such that Xi > yn. Since this 
implies (/ - l)n + fjLi + 1 G t(A) fl (fe) (/i), we obtain t(A) fl0 (fc) (yu) 7^ 0. Thus we have 
<V t(A) (A), = from (E3]). □ 

Let s and n be positive integers, and let A = (a i j)i< i < s+n _i j i<j< sn be an (s + 
n — 1) x sn matrix. We arrange the column vectors V L ( p ){A) (/i G ^ sn ) into an 
CT 1 ) x CT 1 ) matrix .M( A) : 



.M(A) = (V tM (A)) 



'[s+n-in 



(2.6) 



where the indices /i are arranged in the increasing order with respect to <. For 
example, if s = 3 and n = 2, then 



(detAf 2 detA\l detA\l det Af A 



M(A) 



l 12 

detA^ detA\ 3 3 detA^ det 
detA}f detA** detA** det A** 

23 
1 34 



detAjl detA?? detA?? detA 



l 13 



15 



detA| 
detA" 
detA^ 
det A|| 



det AU\ 
det AH 
det A\t 
det A|| 



detA?* detA?* detA? 4 det A^ 
VdetAff detA? 4 detA? 4 det A^ 



det Ai| det AH 
det AH ^ AU 



det A^ J 



Then our goal Eq. ( 1 1.5]) is to prove 



det .M(A) = JJ det A j, 

Jec° 



(2.7) 



where C° = \ fi G Z° J. 



Given an injection 



s - 1 



we consider an x ) x ( s+ ™ x ) matrix defined by 
M{*,A) = (V. W (A)) = ((-l) |7| -^ +1)/2 detAi M 



/6 



'[s + n-l]\ 



(2.8) 



(2.9) 



where the indices /1 are arranged increasingly with respect to the order defined above. 
Then, from ( 12. 5ft . we obtain 



det M (A) • det M(<&, A) = det fVW (A)M ($, A) 

= det«V l(A) (A),V, M (A)» A ^ 

= det (e(i(A), $(//)) det A t(A)u$(/l) ) A ^ . 



(2.10) 
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In the next subsection we choose appropriate maps $ which enable us to compute 
the last determinant in (I2.10p . Since Q[ay |1 < i < s + n — 1, 1 < j < sn] is a unique 
factorization domain, the irreducible factors of det M. (A) must appear in the last 
determinant. 

2.3 Finding factors of the determinant 

The goal of this subsection is to prove (12. 7p up to constant. In this subsection, we 
work in the polynomial ring S = Q[flij |1 < i < s + n — 1, 1 < j < sn] over the 
rational number field Q in the indeterminates a^-'s. 

First we prove the following lemma which says each irreducible factor of det M. (A) 
is of the form det A^ u ), v G Zg S+n _ v 

Lemma 2.3. Let s and n be positive integers. Let A be an (s + n — 1) x sn matrix. 
Then there exist non-negative integers m u , v G Zg S+n _ l7 and a constant c G Q such 
that 

detM(A) = c Y\ (det Ah)™" (2.11) 

holds. 

Proof. Let 7r : Z s n — >■ C = {1, 2, . . . , s} be a map determined by the following 
condition: 

For G -Z^n, the color k = vr(/i) is the least index satisfying fi^ = 
max{/i( | / = 1, . . . , s}. 

Note that, if 7r(/i) = k, then > and /i G P^. 
Define a map $ in (12. 8j) by 

= W ^(/i) for /iGZ s , n , 

where 0( fe ) is as defined in (12. 2p . Then we claim that the determinant in ( 12 . 1 Of) 
equals 

(2.12) 

If we prove (I2.12p . then we obtain the desired result (12. lip since t(/i) U G 
t(Z° s+n „ 1 ) for any k from Proposition I2.1[ and det Aj, J G (J^Ij), are irreducible 
polynomials in the unique factorization domain 5*. 

To prove (12 . 12j) . set m = m s ^ n = n — \n/s], where \x] denotes the greatest 
integer which is less than or equal to x. Then m satisfies < m < n. We set 

Qi = {fi= (/ii, ...,li s ) G Z Sin | max{/i/ 1 1 = 1, . . . , s} = n - i) (0 <i <m). 

Then we have Z S)Jl = Qo U Qi U • • ■ U Q m . For example, if s — 3 and n = 2, 
then we have m = m 3i2 = 2 - [2/3] = 1, Q = {(2, 0, 0), (0, 2, 0), (0, 0, 2)} and 
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Qi = {(1, 1, 0), (1, 0, 1), (0, 1, 1)}. Then we claim that the determinant on the left- 
hand side of (I2.12p satisfies 



det((V l{x) (A),V m (A))) X4ieZsn 

= ± J] detA lMu${/i) .det((V 4(A )(i),V $M (A)}) Ai/j6Q>r 



(2.13) 



MeQ<r 

for < r < m + 1, where 



r-l 



Q<r — | | Qi, Q>r — | | Qi 



i=0 i=r 

We prove this identity ( 12 . 1 3[) by induction on r. If r = then ( 12 . 1 3[) is triv- 
ial. Assume (12. 13[) is true for r = t. If A G Q>t and ji G Qt, then A& < n — t 
and /ifc = n — t where k = vr(yu). Thus, if (V t (X)(A), V^^r^A)) ^ 0, then 
A ^ from Proposition 12. 2[ which implies A = [i. By expanding the determinant 
det ((V L (\)(A), V$( M )(A))) A Atg g >t along /ith column for /i G we obtain that this 
determinant equals 

± J] det A^ )um ■ det ((V t(A )(A), V* M (A)» A ^ eQ ^ +i , 

which implies (12. 13j) is true when r = t + 1. Considering the case r = m + 1 in 
(I2.13p . we obtain (I2.12p . This proves our lemma. □ 

Example 2.4. If s — 3 and n = 2, then the map 7r : Z 3 2 — >■ C = {1, 2} is given by 

tt(2,0,0) = 1, 71(1,1,0) = 1, 7r(l,0,l) = l, 
tt(0,2,0) = 2, tt(0,1,1) = 2, tt(0,0,2) = 3. 

Thus, the map $ : Z^ 2 -»■ (f) ($(//) = Wm)) (aO) is visualized as follows: 



(2,0,0) m- 
(1,0,1) m- 
(0,1,1) m- 



1 


2 


3 


4 


5 


6 


(1,1,0) H- 


1 


2 


3 


4 


5 


6 






















o 











O 
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(0,2,0) m- 


O 













C) 










C) 


(0,0,2) ^ 


C) 




o 









Then the 6x6 matrix A) in (12. 9ft is given by 



M{$,A) 



( det A\i 


det AH 


det Af* 


det AH 


det AH 


det AH \ 


-det AH 


- det AH 


- det Af 6 


- det Ajl 


- det A?* 


- det A?| 


det A§ 


det A|f 


detAH 


det 


det Ajl 


det A? 3 , 


det AH 


det A\l 


det A|* 


det A}* 


det A** 


det A^ 


-detAH 


- det A* 3 , 


-detA^ 


- det A\l 


- det A\l 


- det A\l 


\ det^i 


det A\l 


detA^ 


det A}2 


det Ai| 


det A}2 
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Hence we obtain that 



t M(A)M(^,A) 

= ((V t (A)(A),V $M (^))) AiMeZ32 

/ det A1235 det A 1245 det A 123fi 
det A1345 











V 










- det A 1356 










det A1345 









det A 1U6 
det Ai 356 










detv4i 356 y 



whose determinant is equal to 

- det A1235 (det A 1U5 ) 



(det A 



1356 J 



Lemma 2.5. Let s and n be positive integers. Let A be an (s + n — 1) x sn matrix. 
Then there is a constant d £ Q such that 



det M (A) = c Yl det A l{y) . 



(2.14) 



Proof. We may assume n > 2 without loss of generality. First, we claim that 
the integers ■m u , v £ Z s ° n , in (12.111) must be or 1. To prove this claim, we choose 

another map $ in (12.81) and use the corresponding x matrix A / l($, A) 

defined in (12.91) . Fix a color k £ C. In the first step of our proof we construct a 
map $ : Z sn — > f^j) whose restriction to p( fe °) = Z Sjn \Pn agrees with </>( fe °) given 
by (12.21) and which satisfies the following three conditions: 

(i) for any A, /i £ Z s>n , we have (V L (\)(A), V$ (At) (A)) = unless A ^ fco /i, 

(ii) <,(//) n $(/i) = for fi £ Z a , n , 

(iii) if n £ pf o) , then t(/x) U $(/i) £ Z s °, 



s,s+n— 1" 



To construct such a map we divide P^ into two subsets. Recall that fi^' is the 
composition whose /th entry is n and others are 0. If we put = {f$ \l^k } 



,(0 ;< 



then P( fc °) is an (s — l)-element subset of Pi . We set the value $(/x) for /i £ P„ 
as follows: 

(a) if fj, £ Pf o) \ P (fco) , we define $(//) = . 

<w = u {(«' 

l<i<s 



>(ko) 



by (E2D, i.e. 

- l)n + 1}, 
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(b) if $ G i? (fco) (I ^ k ), we set 

l<i<s 
i^ko, I 

To see that the condition (i) is fulfilled, it is enough to consider the case when 
\l G Pn . If /i G Pn k °^ \ R( k °\ then the proof is exactly the same argument as we 
did in the proof of Proposition 12.21 If A j^k f 1 ^ (I ko), then there exists i ^ ko, I 
such that Aj > 0. Thus we conclude from ( 12.151) that ^(n^) fl t(A) ^ 0, which 
immediately implies (V L ^)(A),V^)(A)) = by <I575]> . This proves (i) for /i G i? (fco) . 
The condition (ii) is almost obvious from the definition. The condition (iii) is also 
fulfilled since (k - l)n + 1 £ and (jfe - l)n + 2 G $(//) for // G Pi fc[,) . 

Now we claim that the determinant in (I2.10p equals 

(2.16) 



It follows from Proposition 12.11 that 



] [ det A(At)u$( M ) = ] ] det Ah, 



where 

n-l 

p^o) _ | | p(ko) 

i=0 

and from (ii) and (iii) that □$(//) G" Z° s+n _i and det A L ^ u<i> ^ ^ for /i G PjT 
Thus, if we prove (I2.16p . then we can conclude that the integers m v , v G Z® s+n _ x , 
in (12.1 ip must be or 1 since S = Q[ay] is a unique factorization domain. 
To prove (I2.16p . it is enough to prove that 

det«V t(A) (^),V $(M) (A)» AMeZsn 

= ± II det A( M )u$M-det(<V t(A )(A),V $(M )(A))) A ^ p ,, 0) (2.17) 

holds for < r < n + 1, where 

r— 1 n 
p^o) _ | | p(fco) p^o) _ | | p(fco) 

i=0 i=r 

But, this can be proven by induction on r exactly in the same way as we proved 
(I2.13P in the proof of Lemma 12.31 

Lastly we need to show that m v = 1 for all v G Z® n . In fact one easily sees this 
by comparing the degree of the both sides of (12. lip . The left-hand side det M.(A) is 
a polynomial of degree ^( s+ ™ -1 ) , while the right-hand side is a polynomial of degree, 
at most, (n + s — l)( s ^™~ 2 ). Thus all m v must be 1. This proves our lemma. □ 
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Example 2.6. If s = 3, n = 2 and fco — 1> then we have 

PW = {(2, 0, 0), (1, 1, 0), (1, 0, 1)}, = {(0, 2, 0), (0, 0, 2)}, 

and the above $ takes values: 

$(2, 0, 0) = {3, 5}, $(1, 1, 0) = {4, 5}, $(1, 0, 1) = {3, 6}, 
$(0, 2, 0) = {2, 5}, $(0, 1, 1) = {4, 6}, $(0, 0, 2) = {2, 3}. 



(2,0,0) 
(1,0,1) 
(0,1,1) 



i — Y 
i — ¥ 
i — ¥ 



1 


2 


3 


4 


5 


6 




















o 






o 








o 








1 2 3 4 5 6 



(1,1,0) H- 
(0,2,0) ^ 
(0,0,2) m- 









o 









o 






C) 









o 









Thus the 6 x 6 matrix M(<&, A) in (I2.9P equals 



/ det AH 


det AH 


det AH 


det AH 


det AH 


det AH \ 


-det AH 


- det AH 


~ det AH 


-det AH 


- det AH 


- det AH 


det Af 5 


det AH 


det AH 


det Af 5 


det AH 


det AH 


det AH 


det A\l 


det AH 


det AH 


det AH 


det AH 


-det AH 


- det A\l 


-det AH 


- det AH 


- det AH 


- det AH 


\ det AH 


det AH 


det AH 


detA% 


det AH 


det AH / 



Hence we obtain that 

l M{A)M{^,A) 

= det«V t( A)(A) ) V. w (i4)» Af 



/ det A 1235 





V o 



det A1245 
det A1345 











det A 1236 


- det A 1356 








- det A 1235 


det A 2U5 





det A 12m 
det A 13m 

- det A U56 



- det A U56 










\ 



det A 1235 



det A 2356 J 



whose determinant is 

det A 2U5 det A U56 det A 2356 

2.4 Determining the constant 



[ [ del .1,;,,.. 



Now we are in position to prove that d — 1 in Eq. f)2.14p . For this purpose we 
specialize the entries of A = (ajj)i<j< s + n -i, i<j<sn as 



'1 < i < s + n — 1, 1 < j < sn). 



(2.18) 
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Here we introduce the lexicographical monomial order (or lex order) on the set 
of monomials in the variables x\ ^> ••• x sn , i.e. a;} 1 • • • x*^ x^---x^ if 
H — jii ■ ■ ■ -,ik-i — jk-i and ik > jk for some 1 < k < sn. If / is a polynomial 
of Xi, . . . ,x sn , then we define the leading coefficient of / to be the coefficient of 
the greatest monomial in the lex order. We also define the leading term of / to 
be the leading monomial including its coefficient. For example, the leading term of 
/ = 4x1X3X3 + 4xg — 5x^X2 + 1 x\x\ is — 5xfx 2 and its leading coefficient —5. If / 
and g are two polynomials with the same leading term, we shall write / ~ g. In the 
above example, we can write / ~ — 5xfx2- If / = {i\ < • • • < i n } E7Z = (' s+ ™ 
and J — {j\ < • • • < ] n } E C = {t(o) \ fi E Z SjTl }, then the leading term of the 



determinant 



detA' =det(xf n ~^ =det 



I XjT ■ ■ ■ x i \ 

Jl Jn 



\ jl ' jn / 



is obviously the product of its diagonal entries which equals x s ^ n n ■ ■ ■ x s ^ n ln . Here- 
—He Xj +n_/ for x s + n ~ h • • ■ x s + n ~ in . Because of det A*j ~ Xj +n_/ , we 



conclude that 

det (detA$) l6W>JgC ~det kV" '),.*..,.,• 

Thus, if we prove the following lemma, then it shows that the leading coefficient of 
det (dety4j) /g ^ J( _ c equals 1 under the specialization ( I2.18p . 

Lemma 2.7. The leading term of det { x j +n I ) Ien JgC is given by the product of its 
diagonal entries. More explicitly, we have 



k=ij=i 

and the leading coefficient is 1. 



k=\j=\ 



Proof. We proceed by double induction on s and n. In this proof we write lZ s>n for 
TZ and C Sjn for C. If s = 1 then the determinant equals and if n = 1 

then we have 

(4 +n - 7 W J& c = det W^W. ~ ^ • • 

This implies that our claim is valid for s — 1 or n = 1. 
Assume s > 1 and n > 1. We put 
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where 

Z fl = \P = Ol, • • • , Vs) e Z S)n I >Ui > 0} , 

z i]l = {p = (vi, ■ ■ ■ , Us) e I //i = 0} . 

Then we have = U C s , n = d°2 U eg, = #d°2 = (t^ 2 ) 

and ifTZ^l = #Q*n = ( s+ ™~ 2 )- Further, under the lexicographic order we defined 
in Section 1, the subset 7£i°n consists of the first ( s ^™7 2 ) elements of 7Z S}n , and Ci°« 
consists of the first ( s ^™7 2 ) elements of C SjTl . Expanding the determinant along the 
first ( s ^™7 2 ) columns, we obtain 

X 

where X runs over all ( s ^™~ 2 ) -element subsets of H s ,ni an d £x = ±1- The symbol X 
stands for the complement set of X in 7?- S)n . Since the variable x\ does not appear 
in det (x s 7 +n_/ ) = (i) , the exponent of X\ attains its maximum when we take 

\ ** ' 1 t-i-j J few's fi 

X = 7^i°n, in which case we have e„(o) = 1. Thus we obtain 



Since we can naturally identify 7?i*n and Cg with TZ s -i in and C s _i in respectively, we 
obtain 

k=2j=l 

(2.20) 

by the induction hypothesis. On the other hand, since each entry of det (^ j +n_/ ) Je7J (o) 

has the factor x{ +n ~ 1 , we can sweep out the factor from each row and the remaining 
determinant has the form of type (s, n — 1) with the s(n — 1) variables X2, ■ ■ ■ , x n , 
x n +2, ■■■,x 2n , ■ X( s _i) n+2 , • • • , x sn -i- Thus we obtain 



s n— 1 
0=2 k=2 j=l 

(2.21) 

It is an easy computation to check Eq. ( I2.19p by using (I2.20p and (I2.2ip . This proves 
our lemma. □ 

Now we can finish our proof of Theorem 11.31 

Proof of Theorem 11.31 Now we consider the right-hand side of Eq. ( 12. 14ft under 
the specialization (I2.18p . For J = {j 1 < ■ ■ ■ < j s + n -i} £ C°, the leading term of 
det A j is obviously the product of the diagonal entries, which shows 

det Aj ~ x ^~l^ = x ^ x s+n-2 . . . 
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Thus it is obvious that the leading coefficient of Y\,.^ 7 a det A L < V \ equals 1, which 
implies d = 1 in ( 12.141) . In fact, by a direct computation, one can also obtain 

We complete the proof of the main formula (11. 5ft . □ 



3 An application to determinants for classical 
group characters 

In this section, we apply Theorem 11.31 to determinants involving classical group 
characters and Macdonald polynomials. We almost follow the notation in [7J. 

A partition is a sequence A = (Ai,A 2 ,...) of non-negative integers in weakly 
decreasing order and containing only finitely many non-zero terms. The non-zero 
Aj are call the parts of A. The number of parts is the length of A, denoted by 
£(X), and the sum of parts is the weight of A, denoted by |A|. For two partitions 
A = (Ai, . . . , A m ) and /i = (/^i, . . . , /x m ) with at most m parts, we say A precedes /i 
in the reverse lexicographic ordering if the first non- vanishing difference A, — is 
positive. 

If A; is a non-negative integer, let 5(k) = (k, k — 1, k — 2, . . . , 1, 0, 0, . . . ) denote the 
staircase partition, and if k is a positive half-integer, let S(k) = (k, k—1, . . . , 3/2, 1/2) 
denote the decreasing sequence of length {2k + l)/2. When X = (xx, . . . ,x n ) is an 
n-tuple of variables, and a = (a>i, . . . , a n ) is a sequence of integers or a sequence of 
half-integers, we use the notation 



V(a;X) = {x?) 1<ii<n , W±{a;X) = (/" : V'), 



<i,j'<n' 



We now recall Weyl's character formula. If A = (Ai, . . . , A ra ) is a partition of length 
< n, we put 

_ detV(X + 5(n-l);X) 
GMA ' X) ~ detV(5(n-l);X) ' (3 ' 1} 

detW-(X + 6(n);X) 
SP2n[K X) ~ detW-(5(n);X) ' (3 ' 2) 

° 2 " +l(A; X) ~ detW-(5(n-l/2);X) ' (3 " 3) 

r detW + (X + 6(n-l);X) 

fA . X x _ (1/2) det W+(5(n - 1); X) " ^ ' , , 

U 2n (A, A ) - < det + ^ _ i) ; X) ^'^ 

1 detW+{6{n-l);X) n ~ 

Then GL n (A; X) (resp. Sp 2 „(A; X)) gives the irreducible character of the general lin- 
ear group GL n (resp. the symplectic group Sp 2n ) with highest weight A. GL„(A; X) 
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is usually called the Schur function. Similarly, Oat (A; X) is the irreducible character 
of the orthogonal group with "highest weight" A. Weyl's denominator formula 
reads as follows. 



Proposition 3.1. 



det V(S(n - 1); X) = j ] (a* - Xj), 

l<i<j<n 



n 



detW-(5(n-l/2);X) = (-irH^ } \ 



i=l A j l<i<j<n 



detW-(6{n)-,X) = {-l) n f[?—£ J] (Xj ^ 



nf . * * / >"» . / >"» . 

1=1 l<l<J<n 



detiy+(5(n-l);X) = 2 TT ^ 

l<i<j'<n J 

In what follows, we consider sn variables xf\ . . . , x« , x*[\ . . . , \ . . . , xf \ . . . , in', 
and we write 

Y - ( r (l) t-W r (2) r( 2 ) r (s) r( s h 
for /i G ^ S) n, We apply our main formula ( II. 5p and obtain the following theorem. 
Theorem 3.2. Let s and n be positive integers. Then we have 

n f s + n-i-j-\\ 

de t (GL„(A;X p ,) Ac((s im uez ^ II. < ^ 

and 

det (0 2 n+i(A;X^) ) = det (Sp 2n (A;X p 



det (o 2n (A;X p 



Ac((s-i)™),AteZ s ,„ V / Ac((s-l)"),/*e2 a 

Ac((s-i) n ),Me^,„ 



(-71 — i — j — 1A 



- n n p ; • < 3 6 > 

l<fc<Z<sij'=l L X i X j J 

where the rows are indexed by partitions A = (Ai, . . . , A n ) such that Ai < s — 1 and 
arranged increasingly in the reverse lexicographic ordering. 

Remark 3.3. If we substitute 

xf ] = t^ak (1 < k < s, 1 < j < n) 

into ( 13. 5 p and ( 13. 6p , we obtain the (symplectic) Schur function identities in [2j Corol- 
lary 3.1, Proposition 7.1]. 
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Proof of Theorem 13.21 We prove one case, say the symplectic case of (13. 6p . be- 
cause the other cases can be proven similarly. For simplicity, set 

1 -u 2 (v-u)(l-uv) 

C{u) = , D(u,v) = . 

u uv 

We apply the formula (jl.5p to the matrix A = (a^) with entries given by 

{k) y+n-i , {k)X -s-n+i 



Q>i,(k-l)n+j — \ Xj J ~ \Xj 

(l<i<8 + n-l,l<k<s,l<j<n). 
Then the (J, J) entry of the matrix on the left-hand side of (jl.5p is equal to 

A ^ • Sp 2 „(A;X^) 

where I and J are given by 

I — {s — Ai, s + 1 - A 2 , . . . , s + n - 1 - A n }, J = 

and 

A .=nn^ (fc) )n n ^vf) n nn^*?)- 

k=l i=l fc=l l<i<j<Hk l<k<l<s i=l j=l 

Hence we see that the left hand side of ( II. 5p becomes 
J] A M -det(Sp 2 „(A;X M )) 

The exponents of C(a^"), D(x\ k \xj) (i < j) and D(xf\x^) are given by 

's + n — i — 1^ 
n — i 

' s + n — j — r 

n - j 
's + n — i — j — V 
n — i — j 

respectively. On the other hand, the right-hand side of (jl.5p becomes 

n a - 

and the exponents of C(x,[ ), D(xi k \xj) (i < j) and D(xf\x^) are given by 
e Z°s+n-l | /^fc > i} — #Z° s+n _ 1 _( i _ 1 ^ = #^° s+n _j = f n _ ^ ^ ' 

#1^ ^ ^s,s+n-l I Mfc — j} — #^s,s+n-l-(j-l) = #^s,s+n-j = ^ 72 — j / ' 

n-i- j + 



e | fi k > i} = #z s , n _i = 
e | ii k > j} = #z s , n -j = 
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respectively. Therefore we see that det ( Sp 2 (A; X„) ) equals 

s n s 

nnc(-f ) ) ( "» : " i »-<' + » : "'»n n ° 

k=l l<i<j<n 

n 



w^t'M^t 1 ) rr n n( x ?\xf )( s+ :t 1 )-( s+ :t 1 ) 

k=l i=l fc=l l<«<j<n 

n 

X 

KM<s i,i=l 



= ii n 

l</c<Ks i,j=l 

This proves the symplectic case. 

One can prove the odd orthogonal case similarly by merely taking 

Sk)\ s+n - i - 1 ' 2 _ ( jk)y s - n+i+1/2 

(l < i < s + n - 1, l < k < s, l < j < n) 



a i,(k-l)n+j ~ \ X j ) ~ \ X j 



and C(u) = — u~ l l 2 {\ — u) in the above arguments. 

One should be more careful to prove the even orthogonal case because of the 
multiple 2. If one takes 

fl - a n ■ - (V fc) v +n_i-1 + r>)r s_n+i+1 

«i,(fc-l)n+j — I x j ) \ \ x j J 

(l<i<s + n-l,l<k<s,l<j<n), 
then the (I, J) entry of the matrix on the left-hand side of (II. 5p is 

An ■ 2n (A; Xp), 

where 
A, 

m=i rw,^ D{ x f\xf) u 1<k<l <s mi n?=i D{4 k \*?) if ^ o, 
.2 mux rii^-^ ^(4 fc) , *f ) rL<*<i<- n& ny =1 ^(4 fc) , 4°) if A « = °> 

/s+n-2\ 

which implies the left-hand side have the multiple 2\ /. On the other hand, each 

/s+ra-2\ 

determinant on the right-hand side has the factor 2. Hence 2v / also appears on 
the right-hand side. 

Considering the principal term of asymptotic behavior of the both sides of (13. 6p 

at xf^ — >■ +00 (1 < k < s, 1 < j < n) (or just taking ai^k-i)n+j = {^f^ )> 
we immediately obtain (13.5j) . □ 

Next we give a similar determinant identities for Macdonald polynomials. Here- 
after we also use a symmetric function / in countably many variables x = (x\, X2, ■ ■ ■ ). 



19 



(See [8] for detailed exposition on symmetric functions in countably many variables 
and Macdonald functions.) The symmetric function p r = Yli>i x i * s called the rth 
power-sum symmetric function. The function 

m A (x) = y^x Q 

summed over all distinct permutations a of A is called the monomial symmetric 
function. The dominance order is defined by 

A n •<=>- Ai + ■ • • + Aj < /ii + • • ■ + yUj for all i > 1. 

Define an inner product (u,v) q> t by the values on the power-sum symmetric 
functions by 

e{X) 1 _ A, 
(P\,Pl*)q,t = h,»Z X Y[ , 
8=1 

where z\ = Yli>i i mi mi- with mj = m^(A) the number of parts of A equal to i. The 
Macdonald symmetric functions P\(x;q,t) are defined by the property that they 
are upper triangularly related to the monomial symmetric functions and orthogonal 
with respect to the inner product. That is, they satisfy the following two conditions: 

1. P\(x;q,t) = m\{x) + 52u-<a c \v(.1i t) m n( x ), where the sum is over /i smaller 
than A in the dominance order. 

2. (P\> P/j)q,t = if A ^ /i. 

The Macdonald symmetric functions Q\(x; q,t) are defined by 

Qx(x; q, t) = b\(q, t)P\(x; q, t) 

where 



cSA 

with a(c) — Xi — j the arm-length and /(c) = X'j — i the leg-length at c = G A. 
From Equation (13.51) we immediately have the following: 

Corollary 3.4. Let s and n be positive integers. Then we have 

det(P x (X,-q,t)) Xci{s _ 1)n) ^ Zsn = J] n(^-4°) . ( 3 - 7 ) 

l<k<l<s i,j=l 

j , , n / Y ,\\ _ (t n ; q) s -i(t; t) n -i 

aet^ A ^A M ,g,t;j Ac((s _ 1)n))M6ZaiB - (g . g)si(tgS -i. t)ni 

/ s+n — i— j — 1\ 

x n n(^ , -4") . p-s) 

l<fc<Ks i,j'=l 

where the rows are indexed by partitions A = (Ai, . . . , A n ) such that Ai < s — 1 and 
arranged increasingly in the reverse lexicographic ordering. 
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Proof. The first identity (13. 7p comes from the fact that 

P x (x; q, t) = GL n (A; x) + 2J a\,n{q, t)GL n ((ji; x) 

where the sum is over \x smaller than A in the dominance order. This relation follows 
easily from the above definition and 

GL n (A;x) = yKx^m^x), 

where the Kostka number ^ satisfies K\ jti = unless \i -< A and K\ \ = 1. If 
v G Z Stn , then we have GL n (/x; X v ) = unless £(ff) < n. Hence, for A C ((s — l) n ), 
and v G Z Sjn , we obtain 

P X (X„; q, t) = GL n (A; X v ) + ax >^ ^) GL n(^; 

MC((«-l) n ) 

which immediately implies (I3.7p . We also obtain the second identity f)3.8p from the 
first one ( 13. 7p using 



(t n -q) s . 1 (t-t) n . 



1 



Ac ((s -i)") (q;q)s-i(tq s - 1 ;t) n ^ 1 

This completes our proof. □ 
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